MA085246 


TABLE  OF  CONTENTS 


r 


Section  Page 

1.0  INTRODUCTION  .  1 

2.0  STRUCTURE  OF  MODEL  SYSTEMS  .  5 

3.0  MODELING  QUESTIONS  .  17 

4.0  CONCLUSIONS  AND  RECOMMENDATIONS  .  26 


-li- 


1.0  INTRODUCTION 


This  reporc  represents  the  culmination  of  more  than  two  years 
of  research  on  the  use  of  life  cycle  cost  analysis  in  the  acquisi¬ 
tion  process.  The  specific  concern  of  the  study  has  been  the  abil¬ 
ity  to  perform  cost  tradeoff  analysis  to  find  feasible  least  cost 
mixes  of  capital  and  labor.  A  collateral  goal  has  been  the  develop¬ 
ment  of  methods  capable  of  performing  such  analysis  under  conditions 
of  limited  information  such  as  the  earliest  stages  of  design. 
Accordingly,  the  work  has  resulted  in  two  types  of  developments. 
First,  new  formulations  for  the  estimation  of  manpower  costs  have 
been  developed  and  partially  tested.  Second,  a  new  structure  for 
cost  model  systems  has  also  been  developed  and  tested. 

The  purpose  of  this  report  is  to  present  recommendations, 
based  on  this  research,  for  the  kinds  of  models  and  model  systems 
the  Navy  should  develop  to  support  the  acqusition  process.  A  brief 
review  of  the  reasons  for  the  recommendations  is  presented  here, 
but  the  reader  is  referred  to  earlier  work  for  detailed  rationale.  _ 


*  Five  reports  make  up  the  bulk  of  the  work.  These  are  Neches, 
T.  and  R.  Butler,  "Guidelines  for  Hardware/Manpower  Cost  Analysis," 
AG-PR-A100-2  (3  Volumes),  1978;  Neches,  T. ,  and  R.  Butler,  "Demon¬ 
stration  Model  System,"  AG-PR-A101  (5  Volumes),  1979;  Neches,  T., 

C.  Low  and  S.  Simpson,  "A  Review  of  Current  Manpower  and  Cost  Esti¬ 
mation  Methodologies,"  AG-PR-A103 ,  1979;  Neches,  T.,  C.  Low  and 

S.  Simpson,  "A  Survey  of  Navy  Cost  Model  Input  Variables  and  their 
Data  Sources,"  AG-PR-A104,  1980;  Butler,  R.  and  S.  Cylke,  "Civilian 
Billet  Cost  Model,"  AG-PR-A102,  1979.  In  addition,  see  Neches, 

T.  ,  "The  Demonstration  Model  Systems  An  Overview,"  AG-PM-A104-1 , 
1979  and  Butler,  R.,  "The  Use  of  Life  Cycle  Cost  Analysis  in  Hard¬ 
ware  Design,"  AG-RD-120,  Military  Operation  Research  Society,  44th 
Symposium,  Vandenburg  AFB,  1979. 
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Throughout  the  research  effort,  the  model  system  used  to  study 
various  aspects  of  the  problem  has  been  tailored  to  shipboard  elec¬ 
tronics  systems.  One  of  the  questions  that  must  be  answered,  there¬ 
fore,  is  what  equipment  types  and  operating  environments  must  be 
modeled  separately.  Different,  but  related  problems  have  to  do 
with  the  number  of  maintenance  echelons  and  equipment  indenture 
levels  that  must  be  modeled — and  whether  these  questions  are  related 
to  deployment  environment  or  equipment  type. 

At  another  level,  should  the  idea  of  linked  and  graded  models 
be  used?  If  so,  precisely  what  gradations  are  appropriate  and 
do  these  differ  for  diverse  equipment  types.  Are  general  purpose 
models  feasible  at  all,  or  must  new  models  be  developed  for  each 
specific  system?  Can  multi-environment  models  be  created  by  ganging 

single  environment  models,  or  are  more  complex  adjustments  neces- 

* 

sary?  Are  modular  models  possible,  in  which  the  specifics  of 
a  particular  environment/equipment  combination  can  be  modeled  by 
the  assembly  of  several  standard  modules?  If  so,  how  are  these 
modules  defined  and  implemented? 

The  questions  suggested  above  are  structural  in  nature.  Their 
answers  will  determine  how  many  models  of  what  types  will  ultimately 
be  developed  by  the  Navy  for  use  in  the  acquisition  process.  An¬ 
other  range  of  questions  has  to  do  with  the  algorithmic  content 
of  the  models.  The  broadest  of  these  questions — the  use  of  paramet- 

*  A  multi-environment  model  is  required  for  systems  that  operate 
in  more  than  a  single  environment,  e.g.,  a  laser  communications 
system,  such  as  SAOCS,  linking  surface  ships  and  land  bases  via 
aircraft. 


-3- 


rics  versus  process  formulations — may  be  of  little  import.  Certain¬ 
ly  there  are  instances  where  each  is  appropriate.  This  leads  to 
the  conclusion  that  some  hybrid  form  will  be  the  most  useful,  but 
the  selection  of  the  best  ways  to  use  each  remains  as  an  important 
question. 

Innovations  in  the  estimation  of  manpower  requirements  (and 
hence  costs)  and  spare  stockage  costs  have  been  introduced  by  the 
prototype  models.  These  formulations,  while  logical,  are  untested. 
Their  retention  in  the  final  models  must  therefore  depend  on  the 
results  of  pilot  programs  in  which  reasonably  intensive  testing 
can  be  carried  out. 

Another  significant  and  very  general  question  is  suggested 

by  consideration  of  these  two  areas:  should  cost  models  be  based 

on  optimality  conditions  or  should  they  attempt  to  reflect  the 

★ 

reality  created  by  actual  policies  used  in  the  Navy?  The  latter, 
far  more  realistic  and  extremely  demanding,  would  also  allow  cost 
analysis  of  the  policies  themselves — rather  than  simply  cost  analy¬ 
sis  of  hardware  systems  given  the  policy  environment.  The  virtues 
of  the  former  are  its  ease  and  conventionality:  users  understand 
cost  models  to  be  based  on  optimality  conditions  and  make  implicit 
adjustments  predicated  on  this  expectation. 


*  For  example,  it  was  Navy  practice  some  years  ago  to  determine 
"optimal"  stockage  levels  using  complex  models — and  then  to  purchase 
85%  of  the  amount  determined.  Intended  as  a  means  of  reducing 
spares  cost  by  157.,  there  is  evidence  for  some  systems  that  it 
ultimately  increased  spares  cost,  because  of  behavioral  responses 
to  insufficient  stockage.  Modeling  these  responses  to  estimate 
the  real  cost  impact  of  inappropriate  policy  is  what  we  mean  by 
modeling  reality.  Alternatively,  optimality  conditions  answer 
the  quesion,  how  many  spares  should  the  Navy  buy. 


The  issues  introduced  above  are  discussed  at  length  in  the 
following  pages.  We  have  attempted  to  draw  reasonable  conclusions 
and  base  our  recommendations  on  these  conclusions.  The  criterion 
for  reasonableness  is  the  one  which  underlies,  not  only  the  discus¬ 
sion  here,  but  virtually  everything  written  in  the  reports  cited 
earlier:  cost  effectiveness.  We  have  recommended  a  set  of  model 
systems  predicated  on  our  judgement  about  the  utility  of  a  given 
characteristic  compared  to  its  cost — either  in  development  or  in 
use  by  participants  in  the  weapon  system  acquisition  process. 


2.0  STRUCTURE  OF  MODEL  SYSTEMS 


The  two  major  questions  regarding  the  kinds  of  models  appro¬ 
priate  for  HARDMAN  objectives  are  related  to  form  or  structure 
and  content.  This  section  deals  with  the  first.  During  the  course 
of  research  on  cost  analysis  for  HARDMAN  objectives,  three  structur¬ 
al  issues  have  arisen.  First,  the  idea  of  linked  and  graded  models, 
while  intuitively  pleasing  in  a  laboratory  environment,  may  present 
difficulties  in  execution.  Second,  as  we  think  more  concretely 
about  what  kinds  of  models  are  needed  (i.e.,  electronic  vs.  mechani¬ 
cal  or  sea  vs.  land)  it  becomes  clear  that  a  prodigious  number 
will  be  required.  Third,  because  of  similarities  in  the  general 
structure  and  common  elements  in  many  of  the  model  types  required, 
the  idea  of  modular  cost  model  components  has  sufficient  appeal 
to  be  explored.  These  three  issues  are  discussed  in  the  subsections 
below. 


2.1  LINKED  AND  GRADED  MODEL  SYSTEMS 

The  general  idea  of  linked  and  graded  model  systems  was  intro¬ 
duced  in  the  Guidelines  study  and  explored  further  with  the  Demon- 

ie 

stration  Model  System.  Gradation  of  the  detail  and  attempted 
accuracy  of  cost  models  was  suggested  as  a  means  of  addressing 
the  conflict  between  lack  of  information  at  the  earliest  stages 
of  design  coupled  with  the  profound  impact  of  early  design  decisions 

7 


op.  cit.  A100  and  A101 


on  ultimate  life  cycle  cost.  Existing  models  generally  require 
more  data  than  can  be  generated  in  this  critical  early  period. 

Thus  the  major  contribution  of  graded  models  is  the  introduction 
of  simple  ones  to  provide  some  information  as  opposed  to  nothing 
at  all.  At  higher  gradations  (i.e.,  more  complex  models),  this 
argument  is  no  longer  germane,  of  course.  But,  at  intermediate 
stages  of  design  many  models  demand  too  much  or  too  detailed  infor¬ 
mation.  We  have  observed  many  design  programs,  as  a  result,  in 
which  a  mandate  from  the  government  to  use  a  particular,  detailed, 
model  has  driven  parts  of  the  design  to  a  too-hasty  conclusion. 

That  is,  the  cost  analysis  requirement  has  been  allowed  to  drive 
the  engineering  design,  rather  than  augment  it.  By  introducing 
an  intermediate  level  of  models,  the  grading  system  allows  design 
to  proceed  at  a  natural  pace,  providing  it  with  appropriately  de¬ 
tailed  cost  analysis  during  the  period  when,  although  much  informa¬ 
tion  is  available,  many  aspects  of  the  design  are  still  in  flux. 

Linkage  of  cost  models  follows  as  a  logical  requirement  from 
using  different  grades  of  detail,  and  therefore  different  models, 
in  the  course  of  a  single  design  effort.  The  objective  of  achieving 
linkage  is  to  keep,  as  far  as  possible,  a  consistent  set  of  decison 
criteria.  This  can  only  be  done  by  achieving  mathematical  consist¬ 
ency  between  the  models:  the  algorithms  of  each  model  level  must 
be  essentially  the  same  with  assumptions  replacing  data  in  the 
simpler  versions.  Inconsistent  algorithms  actually  take  substan¬ 
tively  different  approaches  to  the  estimation  of  the  same  thing 
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and  can  therefore  lead  to  inconsistent  conclusions,  given  the  same 
data. 

Existing  models  provide  an  example  of  mathematical  inconsist¬ 
ency  in  the  estimation  of  spares  cost.  The  Air  Force  model,  MOD- 
* 

METRIC,  is  among  the  most  sophisticated  spares  models  in  use. 

It  performs  marginal  allocation  of  funds  to  the  single  item  of 
stock  which  decreases  expected  base  level  backorders  at  the  most 
advantageous  rate  of  cost  for  effect.  This  would  be  analogous 
to  what  we  intend  for  Level  III  models.  To  do  this  type  of  computa¬ 
tion  requires  a  machine  which  can  operate  on  data  representing 
all  of  the  elements  of  a  system  simultaneously.  At  the  other  end 
of  the  scale  (analogous  to  what  we  intend  for  Level  I  models)  are 
sparing  algorithms  typical  of  sequential  process  models  such  as 
the  Air  Force  LSCM  and  the  Navy's  1390B  models.  These  models 
(but  not  the  DMS  Level  I  models)  ignore  cost  criteria  as  well  as 
system  effectiveness  criteria:  they  buy  spares  to  a  stockout  cri¬ 
terion  which  is  independent  of  the  system.  They  do  this,  presum¬ 
ably,  because  they  are  meant  to  consider  one  component  at  a  time, 
thus  requiring  far  less  computational  power  to  run. 

The  middle  ground  between  the  two  extremes  mentioned  (analogous 
to  what  we  intend  for  Level  II)  is  occupied  by  a  wide  variety  of 


*  See,  Muckstadt,  J.,  "A  Model  for  a  Multi-item,  Multi-echelon, 
Multi-indenture  Inventory  System,"  Management  Science,  Vol.  20, 

No.  4,  December,  1973.  By  definition,  a  backorder  exists  when 
there  is  an  unsatisfied  demand  at  base  level. 

**  See,  "Logistics  Support  Cost  Model  User's  Handbook,"  AFALD/ 
XRSC,  Wright-Patterson  AFB,  1979  and  Mil-Std-1390B,  Military  Stand 
ard  Level  of  Repair,  Department  of  the  Navy,  Washington,  D.C., 
1976. 


3 


i 


-8- 


models,  none  of  which  is  representative.  For  our  purpose,  imagine 
a  model  which  purchased  spares  to  a  system  level  effectiveness 
criterion,  but  not  in  the  context  of  cost  minimization.  Inconsist¬ 
ent  formulations  such  as  the  three  just  mentioned  can  lead  to  widely 
different  estimates. 

A  hypothetical  system  is  easily  constructed  which  at  Level 
I  would  require  no  spares,  at  Level  II,  $20  million  worth  and  at 

Je 

Level  III,  perhaps  $15  million.  If  such  a  system  were  designed 
with  the  "help"  of  these  inconsistent  estimates,  one  can  imagine 
that  the  designer  would  soon  become  disenchanted  with  the  whole 
idea  of  cost  modeling.  If  the  design  was  predicated  on  the  results 
of  these  models,  it  would  be  far  more  costly  than  necessary. 

A  set  of  mathematically  consistent  models  would  introduce 
both  cost  and  system  effectiveness  criteria  at  Level  I,  despite 
the  need  to  process  components  sequentially.  This  is  easily  done 
by  developing  a  confidence  level  criterion  which  is  a  function 
of  both  the  individual  characteristics  of  the  component  and  the 
aggregate  characteristics  of  the  system.  A  device  of  this  type 
is  explained  in  detail  in  the  Demonstration  Model  System  (Vol. 


*  These  values  are  based  on  200  systems,  operating  in  200  differ¬ 
ent  locations,  each  with  100  components  with  the  same  failure  rate. 
The  failure  rate  is  just  sufficient  to  yield  a  demand  during  lead 
time  of  .05.  Therefore,  a  (Level  I)  sequential  processor  using 
(as  is  typical)  a  criterion  of  957,  confidence  against  stockout 
would  buy  no  spares.  Another  model  (Level  II)  which  considered 
the  whole  system  at  a  confidence  level  of  95%  (again,  typical) 
would  buy  one  spare  of  each  type  for  each  of  200  locations.  At 
a  price  of  $1000  each,  this  amounts  to  $20  million.  If  there  is 
a  reasonable  variance  among  the  unit  costs  of  the  100  part  types, 
the  use  of  an  economic  criterion  (in  a  Level  III  model)  would  reduce 
the  cost  to,  perhaps  $15  million. 
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I).  That  system  would  yield  a  spares  cost  estimate  of  roughly 
$15  million  at  all  three  levels. 

The  DMS  showed  that  linked  and  graded  model  systems  can  be 
developed.  In  particular,  reasonable  linkage  can  be  created  by 
careful  formulation  of  the  algorithms  used  in  the  lower  level 
models.  But  the  degree  of  detail  in  each  level  is  still  a  question. 

The  terminology  created  to  portray  gradations  was  originally 
machine-determined.  That  is,  Level  I  models  were  originally  con¬ 
ceived  as  those  which  would  fit  on  programmable  calculators,  Level 
II  meant  self-contained  micro  computers  and  Level  III  required 
main-frame  machines.  But  as  the  capacity  of  calculators  and  micro 
computers  grows,  these  distinctions  have  become  somewhat  blurred. 
Even  so,  the  appropriate  level  of  detail  in  the  model  levels  is 
relatively  easy  to  specify.  The  Level  I  models  must  be  usable 
when  no  more  than  five  or  ten  variables  can  be  estimated  to  portray 
a  particular  component.  The  Level  III  models  must  be  capable  of 
accepting  and  exploiting  every  datum  characterizing  even  the  small¬ 
est  nuances  of  a  full  system  design.  Level  II  models  should  be 
as  detailed  as  possible,  given  that  they  must  process  system  compo¬ 
nents  sequentially  rather  than  simultaneously.  These  distinctions 
are  dealt  with  at  greater  length  below. 

2.2  EQUIPMENT  TYPES  AND  OPERATING  ENVIRONMENTS 

The  Navy  procures  a  staggering  variety  of  equipments  for  use 
in  four  dissimilar  environments:  land,  sea,  air  and  space.  The 
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operating  environment  which  must  be  characterized  by  the  operating 
and  support  (O&S)  component  of  a  life  cycle  cost  model  depends 
not  only  on  the  medium  (land,  sea,  etc.),  but  also  on  the  platform 
class  and  sometimes  the  mission.  The  variety  of  possibilities 
is  made  explicit  by  the  following  list: 

Land:  Operational 

Administrative 

Sea:  CV 

Other  surface 
Submarine 

Air:  CV 

NAS 

Space:  Land 
Space 

The  equipments  themselves  can  be  divided  into  three  broad  classes — 
electrical/electronic,  mechanical  and  structural.  Virtually  all 
equipments  include  some  mix  of  all  three.  Strange  as  it  may  seem, 
most  of  the  progress  made  to  date  in  life  cycle  cost  analysis  has 
addressed  only  electronics.  Very  little  is  known  about  failure 
mechanisms  and  how  to  portray  them  for  mechanical  or  structural 
items.  A  similar  situation  exists  with  an  item  not  even  included 
in  the  list — software.  It  is  known  that  a  larger  and  larger  propor¬ 
tion  of  system  acquisition  costs  are  devoted  to  software  develop¬ 
ment.  It  is  less  well  known  but  becoming  obvious  that  software 
O&S  costs  are  also  becoming  significant.  We  leave  the  latter  aside 
for  the  moment.  However,  the  large  and  growing  costs  of  maintaining 
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software  (almost  exclusively  manpower  costs)  may  well  lead  to  con¬ 
cern  on  that  score  at  some  time  in  the  future. 

The  simplest  way  to  determine  how  many  models  are  required 
to  cover  all  the  Navy's  systems  is  to  multiply  the  number  of  oper¬ 
ating  environments  (9)  by  the  number  of  equipment  types  (3)  by 
the  number  of  model  levels  (3).  This  gives  the  staggering  total 
of  81  different  models.  If  one  further  considers  that  most  Level 
I  models  are  usually  model  systems  containing  at  least  three  indivi¬ 
dual  programs,  this  total  rises  to  135.  The  time  and  expense  in¬ 
volved  in  developing  so  many  models  is  clearly  unwarranted  if  alter¬ 
natives  are  available. 

One  approach  is  to  cut  down  the  list  as  much  as  possible  by 
determining  which  operating  environments,  while  different,  can 
be  modeled  with  the  same  algorithms.  For  example,  submarines  and 
other  surface  craft  can  be  modeled  in  the  same  way  with  the  differ¬ 
ences  being  portrayed  by  parameters  rather  than  model  structure. 

The  scope  of  such  combinations  is  limited,  however,  if  an  entire 
model  is  taken  as  the  unit  of  account.  The  feasible  extent  of 
such  combinations  seems  to  be  to  cut  the  list  roughly  in  half — which 
would  still  require  about  65  models. 

Another  approach  is  to  separate  the  models  into  their  compo¬ 
nents:  development,  production  and  the  elements  of  O&S  cost  such 
as  spares,  training,  technical  data  and  so  on.  For  each  of  these 
segments  there  is  considerable  commonality.  For  example,  develop¬ 
ment  and  production  costs  will  use  roughly  similar  algorithms  irre- 
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spective  of  operating  environment.  Spare  stockage  requires  no 
more  than  four  different  formulations  to  cover  the  27  equipment/ 
environment  combinations.  Technical  data  costs  and  inventory  entry 
and  management  costs  will  only  require  single  algorithms  for  all 
the  model  types.  Perhaps  three  or  four  algorithms  will  be  required 
to  estimate  manning  for  the  variety  of  environment  types.  At  each 
model  level,  therefore,  it  may  be  necessary  to  build  no  more  than 
the  equivalent  of  two  or  three  full  cost  models  by  separating  the 
cost  elements  and  treating  each  as  a  building  block  for  a  specific 
equipment/environment  combination.  There  are  some  problems  involved 
in  doing  so,  however.  These  are  discussed  in  the  next  sub-section. 

None  of  the  foregoing  discussion  has  mentioned  the  problems 

associated  with  multi-environment  systems.  At  Levels  II  and  III 

these  problems  are  probably  tractable  since  more  than  one  model 

can  be  used  and  the  results  aggregated  easily.  At  Level  I,  however, 

no  simple  solution  has,  as  yet,  suggested  itself.  The  appropriate 

resolution  of  the  question  will  have  to  await  a  pilot  case  before 

★ 

alternative  methods  can  be  explored. 

2.3  MODULAR  COST  MODELS 

Development  of  a  system  of  cost  element  modules  from  which 
a  wide  variety  of  special  purpose  models  could  be  constructed  has 

*  We  have  developed  one  multi-environment  Level  I  model  system 
to  date — the  Navy  Command  and  Control  System  cost  model.  This 
model  used  separate  data  sets  for  land  and  sea  deployment.  But 
it  was  a  very  difficult,  special  purpose  model  system  and  seems 
inappropriate  as  a  prototype  for  the  HARDMAN  models. 
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both  advantages  and  disadvantages.  The  major  advantage  is  cost 
savings,  as  noted  above.  The  disadvantages  have  to  do  with  the 
difficulty  of  developing  general  structures  which  will  serve  for 
a  wide  variety  of  specific  cases. 

In  addition  to  the  decision  whether  to  take  a  modular  approach 
to  model  development,  an  additional  question  is  whether  the  result 
should  be  put  into  use  as  a  modular  system.  While  development 
costs  can  be  saved  through  modularity,  it  seems  reasonable  that 
the  final  product  should  appear  to  the  user  as  a  special  purpose 
model  system. 

Development 

While  development  of  a  modular  system  is  inherently  less  costly 
than  a  very  large  number  of  specific  models,  each  part  of  a  modular 
system  is  more  difficult  to  implement.  The  main  difficulty  is 
to  insure  that  a  module  is  compatible  with  all  the  uses  to  which 
it  will  be  put. 

Compatibility  must  be  achieved  both  in  the  mathematics  used 
for  a  particular  cost  element  and  in  the  computer  code  by  which 
the  algorithms  are  implemented.  For  example,  there  are  normally 
two  demand  rates  used  in  O&S  cost  elements:  average  annual  demand 
and  demands  during  a  peak  period.  The  latter  is  generally  defined 
as  a  deployment  period  for  a  combat  system,  whereas  the  oeriod 
is  arbitrary  for  a  land-based  operational  system.  Moreover,  a 
land-based  administrative  system  may  not  use  peak  demands  at  all. 
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An  algorithm  for  the  number  of  maintenance  specialists  to  be  trained 
may  be  the  same  for  all  three  environments  except  that  the  dimension 
of  a  variable  for  demand  will  be  different.  All  such  problems 
must  be  foreseen  and  accomodated  to  develop  a  modular  system. 

To  insure  that  the  algorithms  operate  correctly  requires  a 
great  deal  of  care  in  planning  the  detailed  specifications  of  the 
entire  system  before  any  single  system  is  actually  developed. 

Since  planning  computer  code  (especially  with  limited  memory  devices 
like  calculators)  is  an  inexact  process,  the  implied  degree  of 
foresight  is  usually  impossible  to  achieve.  In  operational  terms, 
this  means  that  a  comprehensive  plan  will  require  continual  updating 
as  each  module  is  actually  implemented  in  real  code. 

The  modular  approach  also  requires  that  a  standardized  model 
structure  be  developed,  whether  it  is  actually  implemented  or  not. 
The  general  structure  performs  house-keeping  functions,  establishes 
and  controls  the  operating  sequence,  acquires  data  from  storage 
media  and  passes  it  to  the  blocks  of  code  which  use  it  and  so  on. 

In  a  system  which  was  modular  from  the  user's  point  of  view  a  stand¬ 
ard  structure  would  actually  have  to  be  developed.  Modules  would 
then  be  selected  to  cover  each  cost  element  appropriate  to  a  given 
equipment /environment.  In  a  model  system  which  appeared  to  be 
special  purpose  to  the  user,  the  standardized  structure  would  not 
have  to  be  implemented  except  as  guidance  for  each  specific  struc¬ 
ture.  The  latter  course  is  clearly  less  demanding  while  it  also 
implies  the  need  for  a  larger  volume  of  computer  code. 


k. 
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Use 

The  Cwo  basic  approaches  to  the  use  of  modular  models  are 
more  easily  distinguished  when  compared  from  the  user's  point  of 
view.  True  modular  code  requires  the  user  to  select  and  assemble 
the  series  of  modules  required  by  his  specific  problem.  Special 
purpose  models  (whether  developed  under  a  modular  discipline  or 
as  special  purpose  models)  do  not. 

The  need  to  assemble  modules  has  three  effects.  First  it 
imposes  a  cost,  both  in  resources  and  time  on  the  user.  Second, 
it  is  an  error-prone  process,  allowing  a  user  to  make  inappropriate 
selections.  Third,  however,  it  also  makes  it  unnecessary  to  antici¬ 
pate  every  specific  problem  beforehand.  That  is,  if  a  problem 
arises  that  was  not  anticipated,  true  modularity  allows  a  user 
to  formulate  the  most  appropriate  model  system  available  through 
the  full  range  of  modules. 

We  believe  it  is  feasible  to  take  advantage  of  the  flexibility 
inherent  in  a  true  modular  system  while  avoiding  its  problems  by 
making  the  models  appear  to  be  special  purpose  from  the  user's 
standpoint.  It  would  be  necessary  for  the  HARDMAN  Development 
Office  to  remain  prepared  to  assemble  a  new  special-purpose  system 
from  time  to  time,  as  unforeseen  requirements  arise.  Occasional 
updating  would  be  required  as  well.  While  this  would  normally 
be  a  very  costly  process,  it  need  not  be  if  development  is  based 
on  a  modular  approach.  Any  new  requirement  would  normally  be  satis¬ 
fied  by  assembling  existing  modules  or,  at  worst,  adding  one  or 
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two  new  ones  to  the  entire  collections.  Even  "new"  modules  would 
rarely  be  completely  new.  More  than  likely  they  would  amount  to 
minor  modifications  of  existing  code. 


3.0  MODELING  QUESTIONS 


No  matter  how  efficiently  cost  models  are  built,  or  how  clever¬ 
ly  they  are  presented,  they  should  also  be  capable  of  accurate 
cost  estimation.  But  while  this  desire  for  accuracy  has  been  a 
touchstone  of  the  cost  analyst,  we  believe  it  may  have  been  pursued, 
in  some  cases,  to  the  detriment  of  efficacy.  The  most  accurate 
cost  model  in  the  world  is  a  useless  artifact  if  it  is  not  employed 
for  cost  analysis.  The  structural  questions  raised  in  Section 
2  were  largely  concerned  with  how  to  develop  a  useful  cost  model 
system. 

This  section  explores  three  major  areas  having  to  do  with 
the  accuracy  of  cost  algorithms.  One  are.t — manpower— is  of  obvious 
interest  because  of  the  goals  of  the  HARDMAN  Development  Office. 
Beyond  this,  however,  manpower  is  an  important  area  because  of 
the  general  lack  of  accuracy  with  which  its  associated  costs  have 
been  estimated  by  most  extant  models.  Another  specific  area  treated 
below  is  spares  cost.  Here  the  focus  of  attention  is  the  tremendous 
variation  in  estimates  yielded  by  the  variety  of  existing  models — 
and  the  possibility  of  bringing  early  and  late  estimates  closer 
together.  The  third  area  dealt  with  is  a  more  general  question 
which  potentially  impacts  the  estimation  of  all  cost  elements: 
should  cost  models  portray  optimal  or  real  environments? 

There  are  several  areas  of  cost  analysis  not  addressed  here. 

For  example,  a  wide  variety  of  techniques  is  currently  employed 


for  the  estimation  of  support  and  test  equipment  costs.  These 
costs  are  undeniably  important  and  the  variety  of  methods  produce 
very  large  differences  in  cost  estimates.  Nonetheless,  they  are 
not  treated  here  simply  because,  at  this  point,  we  have  little 
to  add  to  the  field.  Other  cost  elements,  like  transportation, 
can  be  modeled  with  great  complexity,  but  their  importance  is  so 
small  that  they  are  also  ignored.  Neither  of  these  is  the  case 
with  either  manpower  or  spares  cost  estimation. 

3.1  MANPOWER  COST  ESTIMATION 

Manpower  costs  include  those  directly  associated  with  the 
retention  of  a  man  in  the  service,  which  we  (imprecisely)  call 
compensation  costs,  those  associated  with  job-specific  training 
and  a  number  of  other,  indirect  costs.  In  the  algorithms  developed 
for  HARDMAN,  compensation  costs  depend  only  on,  among  other  things, 
the  number  of  hours  of  labor  actually  consumed  by  a  given  hardware 
system.  To  estimate  training  costs,  however,  the  algorithms  first 
establish  how  many  people  (i.e.,  integer  numbers  of  people)  must 
be  trained.  Among  the  various  indirect  costs  some,  such  as  security 
clearance  costs,  depend  on  integer  numbers  of  people  while  others, 
such  as  management  and  administrative  costs,  depend  on  the  amount 
of  a  man's  time  actually  used  by  the  system. 

The  central  mechanism  used  in  the  manning  algorithms  distin¬ 
guishes  three  groups  of  manpower  in  a  ship  environment:  men  avail¬ 
able  with  the  appropriate  "A"  school  background  for  the  skill 


needed;  men  available  with  no  appropriate  school  background,  but 
capable  of  passing  both  "A"  and  "C"  school  courses  of  instruction; 
and  men  in  general  manpower  pools  not  currently  assigned  to  the 
ship.  These  groups  are  called  the  AN,  AG  and  AS  manpower  pools, 
in  ascending  order  of  cost  to  prepare  for  duty.  Members  of  the 
AN  pool  need  only  "C"  school  training  while  those  from  the  AG  and 
AS  pools  also  need  "A"  school  training  to  equip  them  for  duty  with 
the  new  system.  People  drawn  from  the  AS  pool  may  also  need  other 
kinds  of  training  and  give  rise  to  other  costs  because  they  add 
to  the  ship's  company. 

The  algorithms  are  driven  by  the  size  of  the  AN  and  AG  pools, 
filling  the  new  manhour  requirement  as  economically  as  possible. 
While  conceptually  reasonable  (in  fact  a  faithful  portrayal  of 
the  way  in  which  ships  satisfy  the  need  for  new  specialists)  the 
algorithms  impose  data  problems  on  the  model  user  which  may  be 

ie 

impossible  to  overcome.  Conversations  with  experienced  officers 
from  the  fleet  have  provided  a  variety  of  statements  concerning 
the  availability  of  underutilized  personnel.  Most  seem  to  agree 
that  the  force  is  overloaded  with  work  already,  but  many  agree 
much  of  this  work  could  either  be  foregone  entirely  or  performed 
in  far  more  efficient  ways  (i.e.,  labor  substitution  opportunities 
abound  at  very  favorable  rates  of  exchange). 

*  See  the  discussions  of  AG  and  AN  in  op.  cit.  A104,  Appendix 

A. 


-20- 


As  the  algorithms  are  currently  formulated,  the  AS  pool  (shore 
personnel)  are  treated  as  lacking  the  appropriate  A  school  back¬ 
ground.  If  the  available  manpower  on  board  is  as  restricted  as 
testimony — though  not  data — indicates,  then  far  more  attention 
will  be  focused  on  the  AS  group  than  was  intended.  At  a  minimum, 

this  would  imply  the  need  to  split  the  shore  pool  into  groups  with 

* 

and  without  the  appropriate  "A"  school. 

Note  that  these  questions  are  only  pertinent  with  regard  to 
training  costs  and  others  which  require  an  integer  number  of  person¬ 
nel  to  drive  them.  Compensation  costs  are  based  on  the  notion 
of  opportunity  cost:  whether  a  man  is  really  available  (i.e., 
underutilized)  does  not  matter  since  one  of  the  elements  of  his 
compensation  costs  associated  with  the  new  system  is  his  foregone 
output  in  the  alternative  use.  Thus,  if  the  job  he  used  to  do 
is  simply  not  performed  any  longer,  the  foregone  benefit  is  regis¬ 
tered  as  a  portion  of  the  cost  of  his  performing  the  new  work. 

The  budget  cost  arising  from  such  a  shift  can  either  be  equal  to 
or  greater  than  the  cost  charged  to  the  new  system,  depending  on 
whether  the  Navy  gets  along  without  his  former  output  or  replaces 
it  by  adding  more  labor.  In  either  case,  the  real  cost  created 
by  the  new  system  is  the  same. 

As  with  some  of  the  other  questions  raised  here,  the  signifi¬ 
cance  of  the  data  issue  can  only  be  tested  by  an  attempt  to  employ 

*  This  idea  arose  in  a  conversation  with  Dr.  Mike  Sovereign 
of  the  Naval  Post  Graduate  School,  whom  I  would  like  to  thank  for 
this  and  several  other  comments. 
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the  algorithms  with  a  pilot  program.  Alternative  formulations 
can  then  be  explored  if  data  collection  appears  infeasible. 

3.2  SPARE  STOCKAGE  COST  ESTIMATION 

The  spare  stockage  algorithms  introduced  in  the  DMS  for  Level 
1  and  II  models  are  distinguished  by  three  features.  First,  the 
number  of  spares  are  sensitive  to  geographic  location,  making  it 
impossible  to  place,  for  example,  .1  spares  on  each  of  10  ships. 

i 

Second,  the  effectiveness  criterion  employed  allows  each  component 
to  be  spared  with  respect  to  its  relative  cost  and  its  contribution 
to  the  system's  failure  rate.  This  is  done  despite  the  need  for 
sequential  processing  of  each  component  due  to  limited  machine 
capacity.  Third,  the  spares  solutions  are  linked  to  recurring 
unit  production  cost.  That  is,  the  average  unit  cost  declines 
through  a  learning  curve  formulation  as  numbers  of  spares  increase, 
possibly  altering  the  confidence  criterion  to  which  spares  are 
procured  and,  in  any  case,  changing  the  total  cost  of  spares  and 
production  of  the  system. 

Because  the  algorithms  used  for  sequential  processing  employ 
the  techniques  described  above,  they  are  able  to  simulate  solutions 
formerly  obtainable  only  with  large  machines  which  allowed  simulta¬ 
neous  consideration  of  all  system  components.  As  a  result,  the 
spare  stockage  solution  obtainable  by  aggregating  Level  I  model 
solutions  early  in  the  design  process  should  not  differ  markedly 
from  those  obtainable  at  the  end  of  the  development  process — except 


as  a  result  of  data  refinement. 
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The  algorithms  used  in  the  DMS  are  based  on  a  Poisson  arrival 
distribution  or  a  normal  approximation,  implying  a  (stationary) 
exponential  failure  mechanism  for  electronic  components.  While 
much  evidence  is  available  to  indicate  that  these  probability  dis¬ 
tributions  are  appropriate,  evidence  also  exists  to  the  contrary. 

We  prefer  to  make  no  judgment  about  the  relative  virtues  of  this 
or  that  distribution.  The  methods  outlined  for  simulating  simulta¬ 
neous  solutions  on  sequential  processors  will  hold,  no  matter  what 
distributions  are  used.  These  methods  are  recommended  for  use 
by  HARDMAN  in  the  development  of  new  model  systems. 

3.3  MODELING  REALITY  VERSUS  OPTIMALITY 

An  intensely  interesting  and  difficult  question  which  poten¬ 
tially  impacts  all  areas  of  cost  modeling  is  the  choice  between 
realism  and  optimality  as  the  focus  for  model  building.  The  speci¬ 
fic  question  is,  should  a  particular  cost  model  (or  element)  attempt 
to  model  the  costs  which  will  actually  occur  or  should  it  model 

if 

an  "optimal"  process? 

To  illustrate  the  difference,  consider  spare  stockage  estima¬ 
tion.  Most  cost  models  of  which  we  are  aware  (except  those  using 
fixed  proportions  of  acquisition  cost)  base  spares  estimates  on 
an  optimization  routine  that  buys  enough  to  fill  up  a  pipeline 
and  provide  a  buffer  stock  whose  size  is  related  to  the  theoretical 

*  Such  terms  should  be  used  cautiously.  We  mean  quasi-optimal 
or  as  close  to  optimal  as  feasible,  given  constraints  on  model 
size,  running  time  and  so  on. 
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variance  of  the  failure  distribution  and  the  desired  confidence- 

it 

against-stockout.  This  is  an  optimal  system. 

The  alternative  would  recognize  that  spares  will  normally 
not  be  purchased  at  optimal  levels.  The  exigencies  of  budgeting 
dictate  that  systems  will  frequently  be  deployed  with  less  than 
the  optimal  level  of  spares.  Even  if  they  were  optimally  spared, 
the  (real)  non-stationarity  of  failure  rates  will  mean  that,  ini¬ 
tially,  spares  complements  will  be  insufficient  because  of  burn-in 
problems. 

In  the  Navy,  the  initial  lack  of  sufficient  spares  characteriz¬ 
ing  deployment  of  most  systems  has  had  frequently  disastrous  ef¬ 
fects  which  arise  from  behavioral  responses  to  inadequate  supply. 

For  example,  a  maintenance  technician  may  attempt  an  unauthorized 
repair.  This  can  increase  the  failure  rate  if,  as  expected,  he 
does  further  damage,  and  increase  the  demand  lead  time  by  delaying 
entry  of  the  part  into  the  established  repair  cycle.  After  being 
ill-served  by  the  repair  system  (which  usually  requires  a  year 
or  two  to  become  as  effective  as  predicted),  the  technician  will 
begin  doing  things  such  as  ordering  components  he  doesn't  need 
and  refusing  to  turn  in  faulty  components  until  a  replacement  is 
available.  These  modes  of  behaviour  raise  demand  rates  and  increase 

*  A  few  models  operate  from  a  budget  constraint  and  achieve 
as  high  a  confidence  level  as  possible.  Also,  while  related,  a 
number  of  criteria  are  used:  NORS  rates,  backorder  rates  and  avail¬ 
ability  rates  are  examples.  All  such  models  share  the  fundamental 
characteristic  under  discussion:  they  attempt  to  optimize  spares 
levels  given  either  budget  or  effectiveness  constraints. 
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lead  times.  The  net  affect  is  that,  over  the  life  of  the  system, 
a  great  many  more  spares  will  be  required  than  the  original  optimal 
number,  because  the  support  environment  will  have  changed. 

The  reader  will  recognize  that  to  model  the  reality  is  far 
more  difficult  than  to  model  a  theoretical  or  optimal  structure. 

This  is  one,  very  important,  argument  against  modeling  reality, 
no  matter  what  the  circumstances. 

The  cost  of  doing  so  notwithstanding,  there  may  be  other  rea¬ 
sons  to  stick  with  optimal  structures  in  lieu  of  reality.  One 
way  to  decide  would  seem  to  be  a  consideration  of  what  we  expect 
these  models  to  do  for  us.  To  address  that  issue  requires  us  to 
make  a  distinction  between  cost  estimation  and  cost  analysis. 

Every  cost  model  produces  estimates  of  cost.  Very  few  models, 
however,  are  intended  for  use  as  tools  of  analysis:  tools  which 
support  decision-making  by  clarifying  the  relative  cost  implications 
of  different  decisions.  To  be  useful  as  tools  of  analysis,  models 
must  be  easy  to  use  and  responsive  to  the  variables  over  which 
the  user  has  control.  For  example,  a  designer  needs  a  model  that 
is  responsive  to  failure  rates  since  he  can  alter  them,  but  not 
one  that  is  responsive  to  the  number  of  systems  purchased  since 
only  his  customer  can  influence  that  variable  (although  it  is  usu¬ 
ally  convenient  for  him  to  understand  how  quantity  effects  things). 

It  is  basic  to  the  HARDMAN  approach  that  the  most  critical 
decisions  are  the  earliest  ones.  It  follows  that  tools  of  analysis 
are  most  important  in  the  early  stages  of  development.  It  is  also 
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fundamental  that  accurate  planning  be  done  early  enough  to  provide 
a  basis  for  real  resource  allocation  decisions.  The  timing  of 
the  Weapons  System  Acquisition  Process  is  usually  such  that  most 
detailed  allocation  decisions  can  wait  until  close  to  the  end  of 
full  scale  development  and  the  beginning  of  production.  The  impli¬ 
cations  of  these  remarks  are  that  Level  I  models  should  be  good 
tools  of  analysis  while  Level  III  models  should  provide  very  reli¬ 
able  cost  estimates. 

Optimal  structures,  because  they  are  simpler,  are  most  appro¬ 
priate  for  use  in  tools  of  analysis.  By  modification,  especially 
through  addition  of  empirically  determined  cost  estimating  relation¬ 
ships  to  the  basic  structure,  models:  formulated  in  this  way  can 
also  provide  reasonably  accurate  estimates. 

But  cost  estimates  used  to  develop  detailed  plans  and  specifi¬ 
cations  for  the  support  apparatus  should  be  far  more  accurate  than 
those  obtainable  with  optimal  structures.  Therefore,  estimation 
of  costs  in  the  later  stages  of  development  should  be  based,  as 
much  as  possible  on  reality. 

The  problem  with  adopting  these  recommendations  as  the  basis 
for  developing  models  is  that,  where  real  structural  differences 
arise,  this  may  tend  to  violate  the  mathematical  linkage  requirement 
discussed  in  Section  2.  The  resolution  of  this  problem  may  lie 
in  the  use  of  parametric  relationships  to  bridge  the  gap  between 
reality  and  optimality.  This  is  another  research  question  which 
will  have  to  await  the  actual  development  and  testing  of  a  complete 
model  system  (all  three  levels)  which  has  not,  as  yet,  been  done. 
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4.0 _ CONCLUSIONS  AND  RECOMMENDATIONS 


As  Che  foregoing  discussion  indicates,  some  issues  appear 
to  be  far  more  settled  than  others.  In  almost  all  cases  we  have 
reached  definite  conclusions,  whether  some  question  remains  or 
not.  Where  there  is  a  reasonable  degree  of  uncertainty,  the  recom¬ 
mended  course  of  action  reflects  this.  Each  issue  is  taken  sepa¬ 
rately,  the  conclusion  stated  as  succinctly  as  possible,  followed 
by  the  recommended  course  of  action. 

Linked  and  Graded  Model  Systems 

The  research  has  shown  that  linked  and  graded  systems  are 
feasible.  The  use  of  Level  I  systems  by  several  design  teams  indi¬ 
cates  the  acceptability  and  utility  of  the  only  element  in  these 
systems  not  in  common  use. 

Recommendation:  that  HARDMAN  adopt  the  three  level  Linked 
and  Graded  model  system  structure  for  implementation  of  cost 
methods . 

Modularity  of  Cost  Models 

Development  of  the  HARDMAN  models  in  the  variety  required 
to  serve  the  WSAP  can  be  most  effectively  carried  out  with  a  modular 
approach  where  modules  are  cost  elements  or  groups  of  cost  elements. 
The  extra  costs  and  risks  associated  with  presentation  of  the  models 
in  a  modular  form  outweigh  the  costs  associated  with  providing 
the  appearance  of  special  purpose  models  to  the  users. 
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Re  commendation:  that  HARDMAN  adopt  a  modular  approach  to 
the  development  of  cost  models  but  provide  them  to  users  as 
special  purpose  models. 

Manpower  Formulations 

While  the  algorithms  developed  for  early  manning  estimates 
have  desirable  characteristics,  there  is  a  considerable  question 
about  the  feasibility  of  providing  data  to  drive  them.  Nonetheless, 
the  distinction  between  compensation  costs  depending  on  hour-by-hour 
opportunity  cost  of  labor,  and  training  and  other  cost  elements 
dependent  upon  integer  numbers  is  an  important  contribution  to 
cost  realism. 

Recommendation:  that,  initially,  manning  algorithms  be  based 
on  those  developed  in  the  DMS  to  determine  the  severity  of 
data  acquisition  problems  and  to  alter  the  algorithms  as 
necessary  while  retaining  the  essential  structure. 

Spare  Stockage  Formulation 

Level  I  and  II  spares  algorithms  developed  in  the  DMS  provide 
more  accurate  simulation  of  simultaneous  process  formulations  than 
any  others  available  for  use  in  sequential  processing  models. 

Recommendation:  that  HARDMAN  adopt  the  spares  cost  estimation 
techniques  developed  in  the  DMS. 

Reality  Versus  Optimality 

Level  I  models  should  be  developed  primarily  as  tools  of  analy¬ 
sis  and  only  secondarily  as  accurate  cost  estimating  devices. 

Level  III  models  should  be  as  accurate  as  possible  though  still 
capable  of  use  in  cost  analysis.  Optimal  structures  are  appropriate 
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for  cools  of  analysis  while  realistic  models  provide  betcer  estima¬ 
tion.  Parametric  relationships  may  provide  a  means  for  reestablish¬ 
ing  linkage  destroyed  by  the  use  of  these  two  dissimilar  approaches 


Recommendation:  that  Level  I  models  be  based  on  optimal 
structures,  emphasizing  their  use  as  tools  of  analysis  and 
Level  III  models  be  made  as  realistic  as  possible  and  that 
cost  estimating  relationships  be  explored  as  a  means  of 
preserving  linkage  between  the  model  levels. 


